System and method for increasing the accuracy of a medical imaging device

ABSTRACT

A method for improving the accuracy of a digital medical model of a part of a patient, the method includes obtaining a set of at least 2 medical images of the patient, where an element comprising a predefined geometry and/or predefined information was attached to the patient during the recording of the medical images; obtaining at least 2 tracking images taken with at least one camera having a known positional relationship relative to the medical imaging device, said tracking images depicting at least part of the element; determining any movement of the element between acquisition of the at least 2 tracking images; and generating the digital medical model from the acquired medical images, wherein the determined movement of the element is used to compensate for any movement of the patient between the acquisition of the medical images.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. application Ser. No.15/555,502, which was filed on Sep. 2, 2017, which is a national stageapplication of PCT/EP2016/054660, filed on Mar. 4, 2016, and whichclaims the priority of Danish patent application number PA 2015 70121,filed on Mar. 4, 2015. The contents of U.S. application Ser. No.15/555,502; PCT/EP2016/054660; and Danish patent application number PA2015 70121 are all incorporated herein by reference.

FIELD OF THE INVENTION

This invention generally relates to a system and method for increasingthe accuracy of a medical imaging system. More particularly, theinvention relates to the tracking of patient movements during imageacquisition in a medical imaging device, in particular in Cone BeamComputed Tomography (CBCT) scanners.

BACKGROUND OF THE INVENTION

Computed tomography, particularly x-ray computed tomography (CT), is awidely used volumetric imaging principle. In general terms, a radiationsource and a radiation-sensitive image sensor are arranged on a line,with the subject of the examination positioned in between. The subjectattenuates the radiation. The source-detector arrangement is typicallymoved into several positions, often on a circle or segment thereof,around the subject of the examination, and images are taken at everyposition. The spatial, volumetric distribution of the attenuationcoefficient within the subject can then be reconstructed from allimages, for example using the filtered back projection algorithm,generating a 3D digital model of the subject. Often, the image sensor isa 2D sensor, such as in cone beam computed tomography (CBCT). Inmedicine, x-ray CT scanners are valuable non-invasive diagnosticdevices.

One of the major concerns related to the use of CT scanners in medicineis radiation dose. Accordingly, a large body of research has focused onvolumetric reconstruction algorithms that exploit the image data in anoptimal way, allowing fewer images to be taken, or a lower dose perimage, for a given quality of the reconstruction. While filtered backprojection is a direct algorithm, many refined algorithms are iterativeones. Because the volumetric reconstruction problem is ill-posed,various regularization approaches have been suggested, e.g., totalvariation. Maximum-likelihood estimation has also been proposed, forexample with a prior based on material assumptions. Several proposedreconstruction algorithms contain some of the above elements, or all ofthem.

Another way to lower the needed dose in a CBCT system is to make surethe patient does not move during image acquisition. This is because fora given needed accuracy, the signal-to-noise ratio will be greater whenthe patient does not move. Also, when the patient moves, motionartifacts such as for example streaks and aliasing may deteriorate theimage quality. Therefore, in general, the image quality will be betterwhen patient movement is kept to a minimum.

In prior art CBCT systems, various forms of head fixation devices havebeen employed to keep the patient fixated during the x-ray reording.These systems all have the goal of minimizing effects from motion blurand patient movement, thereby achieving a higher accuracy of the finalimages. However, all these systems have the disadvantage that it may beuncomfortable for the patient to be fixated for the duration of thescan, in particular for patients that may suffer from claustrophobia. Ittherefore remains a problem to achieve a high accuracy of CBCT imageswithout having to fixate the patient.

In general, in any image acquisition technique wherein there is thepotential that the target object and imaging device may move relativelyto each other it will be possible to achieve a better image quality in asystem where it is possible to correct for this unwanted movement.

SUMMARY OF THE INVENTION

In one aspect there is disclosed a method for improving the accuracy ofa digital medical model of a part of a patient, the digital medicalmodel comprising at least 2 medical images recorded with a medicalimaging device, the method comprising:

-   -   obtaining a set of at least 2 medical images of the patient,        where an element comprising a predefined geometry and/or        predefined information was attached to the patient during the        recording of the medical images;    -   obtaining at least 2 tracking images taken with at least one        camera having a known positional relationship relative to the        medical imaging device, said tracking images depicting at least        part of the element;    -   determining any movement of the element between acquisition of        the at least 2 tracking images; and    -   generating the digital medical model from the acquired medical        images, wherein the determined movement of the element is used        to compensate for any movement of the patient between the        acquisition of the medical images.

Accordingly, it is thus possible to correct any unwanted movement thepatient makes during acquisition of the medical images when generatingthe digital medical model. The digital medical model may be for examplea 3D model of parts of a patient's head taken with a cone beam computedtomography (CBCT) scanner, but may in general be any medical image ormodel taken over time, that requires a high accuracy. For example, thedigital medical model may also be a 2D medical model such as a panoramicx-ray image or cephalometric x-ray image.

In some embodiments, the predefined information of the element comprisesat least one fiducial marker, such as a plurality of fiducial markers ina predefined pattern, size, shape and/or colour.

When the placement, size, shape and/or colour of the fiducial markersare already known with very high accuracy before any images are taken,it is possible to determine with very high accuracy the movement of theelement between images.

In some embodiments, compensating for any movement of the patientbetween the acquisition of the medical images comprises:

-   -   associating a time stamp with each of the medical images, and        each of the tracking images;    -   determining the position and orientation of the element at each        time stamp and determining therefrom the movement of the element        during medical image acquisition;    -   adjusting the position of each pixel or voxel of the acquired        medical image with an amount corresponding to the movement of        the element.

In this way, it is possible to relate the movement of the element to themovement of the region of interest (ROI) on the patient, and adjust eachof the medical images to account for any movement of the patient duringan exposure of the medical imaging device. Therefore, when generatingthe digital medical model, instead of using medical images where thepatient might have moved between images, the digital medical model canbe generated with images corrected to simulate the situation where thepatient has been still during the entire procedure.

In some embodiments, compensating for any movement of the patientbetween the acquisition of the medical images comprises:

-   -   associating a time stamp with each of the medical images, and        each of the tracking images;    -   determining the position and orientation of the element at each        time stamp and determining therefrom the movement of the element        during medical image acquisition;    -   generating the digital medical model from the acquired medical        images, wherein the generating of the digital medical model        movement of the element is accomplished by iteratively adjusting        the digital medical model to account for the movement of the        element during medical image acquisition.

Iteratively adjusting the digital medical model in this context may forexample mean that a first estimate of the digital medical model is madeusing a computer device, using the acquired medical images before anyadjustment due to the determined movement of the element. Then thedetermined movement of the element for each medical image can be appliedto the digital medical model to iteratively improve the fit of themedical images to account for the movement of the element. So in thiscase, rather than adjusting the position of each voxel or pixel in themedical images and subsequently generating the digital medical model,the digital medical model is generated first, and then iterativelyimproved by using the adjusted position and orientation of the elementand thereby the adjusted position and orientation of the image sensorrelative to the patient.

In some embodiments, the tracking images and the medical images are timestamped using the same clock.

In order to correlate the movement of the patient with the medicalimaging data, it is necessary to be able to map the movement of theelement in time with the recording of the medical data. In principle,the cameras recording the element and the medical imaging sensor couldbe run using two separate processors with each their own clock. However,in this case, the two clocks would have to be synchronized in order tobe able to map exactly the movement of the patient with the medicalimaging data. A simpler solution is to have both the cameras and themedical imaging device run using the same clock. This can beaccomplished for example by running the cameras and the medical imagingdevice from the same computer processor. The computer processor may be astand-alone desktop or laptop computer or any other type of computermeans, or it may be integrated in the scanner.

In some embodiments, determining the position and orientation of theelement at each time stamp comprises:

-   -   recognizing a plurality of the individual fiducial markers in        each tracking image;    -   obtaining a digital representation in a database of the known        predefined pattern and/or shape of the fiducial markers;    -   recognizing the pattern of the fiducial markers in each image to        achieve a best fit to the known predefined pattern of the        fiducial markers on the element from each tracking image.

In order to determine the orientation and position of the element,mathematical optimization algorithms can be used. For example, if thefiducial markers are in the form of dots of a known size, the algorithmscan be used to detect where there are dots and what size they have. Themethod used may for example be principal component analysis (PCA),although other methods are also possible and known to the person skilledin the art.

Since the fiducial markers have a known size, shape and/or predefinedpattern on the element, once the size, shape and position of each founddot is determined, a mask comprising the known predefined pattern of thefiducial markers can by loaded from a database, be overlayed on thetracking image, and the fit of the tracking image to the mask can bedetermined, thereby finding the orientation and position of the element.

In some embodiments there may be more than one camera, such as twocameras or three cameras for recording the movement of the element. Thereason for this, is that if only one camera is used, it is difficult tounambiguously determine how far away from the camera the fiducial markeris. If two cameras are used, it is difficult to unambiguously determinethe position of the element in a direction that is parallel to a lineconnecting the two cameras. If, on the other hand, three cameras areused, possibly but not necessarily, placed for example at the points ofan equilateral triangle, the position of the element in all threedimensions can be unambiguously determined.

Determining the position and orientation of the element using threecameras, can be accomplished for example by having the images from thethree cameras time stamped so that at each time t, there are threeimages taken of the element, recognizing the fiducial markers in eachimage, determining a best fit to the known predefined pattern of thefiducial markers on the element in each image, determining the positionand orientation of the element in each of the three images of theelement at each time stamp, and computing a weighted average of theposition and orientation of the element from the three images. In someembodiments, determining the position and orientation of the element ateach time stamp comprises:

-   -   recognizing a plurality of the individual fiducial markers in        each tracking image;    -   using classification of the indices of the fiducial markers; and    -   matching the known pattern of the fiducial markers on the        element to the pattern of the fiducial markers on the tracking        image using the classification of the indices of the fiducial        markers.

Matching the known pattern of the fiducial markers may for example beaccomplished using a computer device, where the tracking images areloaded, and the fiducial markers are recognized and/or segmented in thetracking images. Then, the position of the fiducial markers in thetracking image are indexed, and the index of the fiducial markers in thetracking image are compared to the known index of the fiducial markerson the element. Since the distance between the fiducial markers on theelement is known, the distances between the fiducial markers in thetracking images can be compared to the known distances, and knownmathematical algorithms can be used to determine the position androtation of the element in the tracking images.

In some embodiments, the camera position and rotation of each camera iscalibrated or determined;

-   -   the intrinsic parameters such as the focal length, skew,        principal point and lens distortion are calibrated or determined        for each camera;    -   the tracking images from the three cameras are acquired        simultaneously such that at each time t, there are three images        taken of the element;    -   the fiducial markers are recognized in each tracking image and        the position of each fiducial marker is determined directly in        the camera co-ordinate frame;    -   the position and/or orientation of the element from the three        images is determined using a cost function to minimise the        difference in the determined position of the fiducial markers in        each of the tracking images.

Since extrinsic parameters of the cameras are known (i.e. the positionand rotation of the cameras with relation to the medical imagingdevice), and the fiducial markers are recognized in each image and theposition of the fiducial markers are determined directly in theco-ordinate frame of the camera, the determination of the position androtation of the element relative to the medical imaging device will bemore accurate.

In some embodiments, the element is attached to a headband, which can beplaced on the patient's head. It is an advantage if the headband isadjustable, since it should be possible to securely attach the headbandto patients with different head sizes such as children and adults,without any risk of the headband moving during the exposure time.

The element may have only one fiducial marker, but preferably shouldhave a plurality of fiducial markers on its surface, for example in theform of dots or circles. There may be any number of fiducial markers,for example more than 10, more than 100, more than 200 or more than 400dots. Preferably there should be enough dots to make it simple to findthe position and size of the dots, but not so many that it would taketoo much processing time.

The adjustment of the medical image may take place substantially in realtime during acquisition of the medical image, or it may be done inpost-processing. The advantage of doing the adjustments duringacquisition is that it is possible to follow the final result of theimage as it is being taken. However, this requires a substantial amountof processing power, so that it may be prohibitively expensive.Conversely, if the adjustment is not done in real time, there is lessneed for high processing power, since the processing can take longerthan the time taken to acquire the medical images.

In some embodiments, there are asymmetrical features on the element orthe element itself is asymmetrical. In principle, it is possible todetermine the position and orientation of the element even if thefiducial markers are all placed in a completely symmetrical pattern. Inthis case, it would be assumed that the element has moved the shortestpossible distance that is consistent with the pattern of the fiducialmarkers, between each time stamp. However, if the fiducial markers areplaced asymmetrically, or if the element itself is asymmetrical, thereis no ambiguity in when overlaying the mask of the known predefinedpattern with the image of the element.

In some embodiments, generating the digital medical model is done inreal time on a computer device while the medical images are beingacquired. This has the advantage that the digital medical model can bevisualized in real time. However, generating the digital medical modelcan in some situations, for example when the digital medical model is a3D CBCT model, require substantial computer processing power. Therefore,it can often be the case that the digital medical model is onlygenerated after all the medical images have been acquired.

The inventive concept of this specification can be used advantageouslyin any medical imaging device where it is important that the patient isstill during imaging, such as standard x-ray, magnetic resonanceimaging, positron emission tomography, etc. However, it is particularlyuseful in CBCT systems where it is very important to get a very highaccuracy of the scan.

In CBCT systems, typical accuracy is in the range 75-350 at the moment,so the accuracy of position determination of the element should be onthe order of the accuracy of the system. Therefore the accuracy of thedetermination of the position and orientation of the element should beat least 400 microns, such as 200 microns, less than 100 microns, lessthan 50 microns, less than 20 microns or less than 5 microns.

In another aspect there is disclosed a method for improving the accuracyof a digital medical model of a patient, the digital medical model beingcomputed based on images recorded with a medical imaging device, themethod comprising:

-   -   obtaining a first tracking image taken with at least one camera        having a known positional relationship relative to the medical        imaging device, said tracking image depicting at least part of        an element comprising a predefined geometry and/or predefined        information;    -   obtaining a first medical image of the patient, where an element        comprising a predefined geometry and/or predefined information        was attached to the patient during the recording of the medical        image;    -   obtaining at second tracking image taken with the at least one        camera having a, said tracking image depicting at least part of        the element;    -   determining any movement of the element between acquisition of        the 2 tracking images; and    -   generating the digital medical model wherein the determined        movement of the element is used to compensate for any movement        of the patient relative to the medical imaging device during the        acquisition of the medical image.

Accordingly, it is thus possible to correct any unwanted movement of thepatient relative to the medical imaging device during the acquisition ofeach medical image. This may be relevant if the exposure time of eachmedical image is longer than the exposure time of each tracking image.For example, in an x-ray system, each medical image taken may require anexposure time on the order of tenths of a second, whereas the trackingimages could have a needed exposure time of only hundredths of a second.In this case, when each x-ray image exposure time is longer than thetracking image exposure time, a plurality of tracking images can betaken during the acquisition of each medical image. Therefore, in thesesituations the digital medical model may be a single 2D image, such as asingle exposure x-ray image. This method is of course not limited tox-ray imaging devices, but any medical imaging device in which theexposure time of the medical imaging device is longer than the exposuretime of the tracking images.

The embodiments described above with respect to the first aspect of theinvention, may also be used in connection with this aspect of theinvention.

In another aspect there is disclosed a medical imaging systemcomprising:

-   -   a radiation source emitting a radiation beam;    -   a radiation sensor for detecting incident radiation from the        radiation beam on a sensor area;    -   an element attachable to a patient, the element comprising a        number of fiducial markers in a predefined pattern, size, shape        and/or colour;    -   at least one camera configured to take tracking images of the        element during the acquisition of medical images taken using the        radiation source and radiation sensor; and    -   computer means for determining a movement based on the tracking        images, and for adjusting the medical images acquired using the        radiation sensor to compensate for the movement.

In some embodiments, the element is attached to an adjustable headband.In this way the element can be made to fit patient's with different headsizes, such as both adults and children.

In some embodiments, the fiducial markers are in the form of circulardots. Dots or circles are simple geometrical features, that are easilyrecognized by computer algorithms.

In some embodiments, the system may include a mouthpiece for helping thepatient stay still during the exposure. The mouthpiece may be in theform of a plate attached to the medical imaging device, and configuredto allow the patient to bite onto the plate.

The element in this specification can be made from any material such asplastic, glass, metal or the like. It is, however important that theelement is made out of a material that is substantially rigid, so thatthe known pattern of fiducial markers will not be distorted over time.

In some embodiments, the element is made of coated glass, and thefiducial markers are printed on the surface of the glass. This materialis both rigid, and it is relatively simple to etch or print fiducialmarkers on the surface of the glass with high accuracy.

In the context of this specification, the term element should beunderstood to mean any device that can be attached to the patient forthe purpose of tracking and determining their movement, and shouldtherefore not be confined to mean only a flat rectangular piece of metalor plastic. In principle, the form of the element could be for examplecircular, semi-circular, pyramidal, triangular, or any other shape. Theelement could even be a complex three dimensional shape, where the shapeof the element itself is used as the fiducial markers.

In the context of this specification, it should be understood thatalthough there may be a reference to a medical image of a patient, thisshould be understood in the broadest sense, so that this also means thatthe medical image can be of only a part of the patient, such as the headof the patient, a jaw of the patient or any other part of the patient.

Embodiments

1. A method for improving the accuracy of a digital medical model of apart of a patient, the medical model comprising at least 2 medicalimages recorded with a medical imaging device, the method comprising:

-   -   obtaining a set of at least 2 medical images of the patient,        where an element comprising a predefined geometry and/or        predefined information was attached to the patient during the        recording of the medical images;    -   obtaining at least 2 tracking images taken with at least one        camera having a known positional relationship relative to the        medical imaging device, said tracking images depicting at least        part of the element;    -   determining any movement of the element between acquisition of        the at least 2 tracking images; and    -   generating the digital medical model from the acquired medical        images, wherein the determined movement of the element is used        to compensate for any movement of the patient between the        acquisition of the medical images.

2. The method according to embodiment 1, wherein the predefinedinformation of the element comprises at least one fiducial marker, suchas a plurality of fiducial markers in a predefined pattern, size, shapeand/or colour.

3. The method according to embodiment 1, wherein the predefinedinformation of the element comprises the geometric shape of the element.

4. The method according to any one or more of the preceding embodiments,wherein compensating for any movement of the patient between theacquisition of the medical images comprises:

-   -   associating a time stamp with each of the medical images, and        each of the tracking images;    -   determining the position and orientation of the element at each        time stamp and determining therefrom the movement of the element        during medical image acquisition;    -   adjusting the position of each pixel or voxel of the acquired        medical image with an amount corresponding to the movement of        the element.

5. The method according to any one or more of the preceding embodiments,wherein compensating for any movement of the patient between theacquisition of the medical images comprises:

-   -   associating a time stamp with each of the medical images, and        each of the tracking images;    -   determining the position and orientation of the element at each        time stamp and determining therefrom the movement of the element        during medical image acquisition;    -   generating the digital medical model from the acquired medical        images, wherein the generating of the digital medical model        comprises iteratively adjusting the digital medical model to        account for the movement of the element during medical image        acquisition.

6. The method according to any one or more of the preceding embodiments,wherein the coordinate system used in determining the position andorientation of the element can be fixed arbitrarily.

7. The method according to any one or more of the preceding embodiments,wherein the tracking images taken and the medical images are timestamped using the same clock.

8. The method according to any one or more of embodiments 4-7, whereindetermining the position and orientation of the element at each timestamp comprises:

-   -   recognizing a plurality of the individual fiducial markers in        each tracking image;    -   obtaining a digital representation in a database of the known        predefined pattern and/or shape of the fiducial markers;        recognizing the pattern of the fiducial markers in each image to        achieve a best fit to the known predefined pattern of the        fiducial markers on the element from each tracking image.

9. The method according to any one or more of embodiments 4- 7, whereindetermining the position and orientation of the element at each timestamp comprises:

-   -   recognizing a plurality of the individual fiducial markers in        each tracking image;    -   using a classification of the indices of the fiducial markers;        and    -   matching the known pattern of the fiducial markers on the        element to the pattern of the fiducial markers on the tracking        image using the classification of the indices of the fiducial        markers.

10. The method according to any one or more of the precedingembodiments, wherein recognizing the fiducial markers comprisesperforming principal component analysis on the tracking images in orderto segment the fiducial markers.

11. The method according to any one or more of the preceding embodimentswherein there are three cameras for recording the movement of theelement, such as three cameras placed at the points of an equilateraltriangle.

12. The method according to embodiment 11 wherein:

-   -   the tracking images from the three cameras are acquired        simultaneously such that at each time t, there are three images        taken of the element;    -   the fiducial markers are recognized in each image;    -   the pattern of the fiducial markers is recognized in each        tracking image to achieve a best fit to the known predefined        pattern of the fiducial markers on the element from each        tracking image;    -   the position and orientation of the element in each of the three        tracking images each time t is determined; and    -   a weighted average of the position and orientation of the        element from the three images is used to determine the best fit        for the position and orientation of the element at each time t.

13. The method according to any one or more of embodiments 11-12wherein:

-   -   the camera position and rotation of each camera is calibrated or        determined;    -   the intrinsic parameters such as the focal length, skew,        principal point and lens distortion are calibrated or determined        for each camera;    -   the tracking images from the three cameras are acquired        simultaneously such that at each time t, there are three images        taken of the element;    -   the fiducial markers are recognized in each tracking image and        the position of each fiducial marker is determined directly in        the camera co-ordinate frame;    -   the position and/or orientation of the element from the three        images is determined using a cost function to minimise the        difference in the determined position of the fiducial markers in        each of the tracking images.

14. The method according to any of one or more of the precedingembodiments, wherein the digital medical model is generated in realtime.

15. The method according to any one or more of the precedingembodiments, wherein the medical imaging device is a cone beam computedtomography device.

16. A method for improving the accuracy of a digital medical model of apatient, the digital medical model being computed based on imagesrecorded with a medical imaging device, the method comprising:

-   -   obtaining a first tracking image taken with at least one camera        having a known positional relationship relative to the medical        imaging device, said tracking image depicting at least part of        an element comprising a predefined geometry and/or predefined        information;    -   obtaining a first medical image of the patient, where an element        comprising a predefined geometry and/or predefined information        was attached to the patient during the recording of the medical        image;    -   obtaining at second tracking image taken with the at least one        camera having a, said tracking image depicting at least part of        the element;    -   determining any movement of the element between acquisition of        the 2 tracking images; and    -   generating the digital medical model wherein the determined        movement of the element is used to compensate for any movement        of the patient relative to the medical imaging device during the        acquisition of the medical image

17. The method according to embodiment 16, wherein the predefinedinformation of the element comprises at least one fiducial marker, suchas a plurality of fiducial markers in a predefined pattern, size, shapeand/or colour.

18. The method according to embodiment 16, wherein the predefinedinformation of the element comprises the geometric shape of the element.

19. The method according to any one or more of embodiments 16-18,wherein compensating for any movement of the patient between theacquisition of the medical images comprises:

-   -   associating a time stamp with each of the medical images, and        each of the tracking images;    -   determining the position and orientation of the element at each        time stamp and determining therefrom the movement of the element        during medical image acquisition;    -   adjusting the position of each pixel or voxel of the acquired        medical image with an amount corresponding to the movement of        the element.

20. The method according to any one or more of embodiments 16-18,wherein compensating for any movement of the patient between theacquisition of the medical images comprises:

-   -   associating a time stamp with each of the medical images, and        each of the tracking images;    -   determining the position and orientation of the element at each        time stamp and determining therefrom the movement of the element        during medical image acquisition;    -   generating the digital medical model from the acquired medical        images, wherein the generating of the digital medical model        comprises iteratively adjusting the digital medical model to        account for the movement of the element during medical image        acquisition.

21. The method according to any one or more of embodiments 16-20,wherein the coordinate system used in determining the position andorientation of the element can be fixed arbitrarily.

22. The method according to any one or more of embodiments 16-20,wherein the tracking images taken and the medical images are timestamped using the same clock.

23. The method according to any one or more of embodiments 19-22,wherein determining the position and orientation of the element at eachtime stamp comprises:

-   -   recognizing a plurality of the individual fiducial markers in        each tracking image;    -   obtaining a digital representation in a database of the known        predefined pattern and/or shape of the fiducial markers;        recognizing the pattern of the fiducial markers in each image to        achieve a best fit to the known predefined pattern of the        fiducial markers on the element from each tracking image

24. The method according to any one or more of embodiments 19-22,wherein determining the position and orientation of the element at eachtime stamp comprises:

-   -   recognizing a plurality of the individual fiducial markers in        each tracking image;    -   using a classification of the indices of the fiducial markers;        and    -   matching the known pattern of the fiducial markers on the        element to the pattern of the fiducial markers on the tracking        image using the classification of the indices of the fiducial        markers.

25. The method according to any one or more of embodiments 19-24,wherein recognizing the fiducial markers comprises performing principalcomponent analysis on the tracking images in order to segment thefiducial markers.

26. The method according to any one or more of embodiments 19-24 whereinthere are three cameras for recording the movement of the element, suchas three cameras placed at the points of an equilateral triangle.

27. The method according to embodiment 26 wherein:

-   -   the tracking images from the three cameras are acquired        simultaneously such that at each time t, there are three images        taken of the element;    -   the fiducial markers are recognized in each image;    -   the pattern of the fiducial markers is recognized in each        tracking image to achieve a best fit to the known predefined        pattern of the fiducial markers on the element from each        tracking image;    -   the position and orientation of the element in each of the three        tracking images each time t is determined; and    -   a weighted average of the position and orientation of the        element from the three images is used to determine the best fit        for the position and orientation of the element at each time t.

28. The method according to any one or more of embodiments 26-27wherein:

-   -   the camera position and rotation of each camera is calibrated or        determined;    -   the intrinsic parameters such as the focal length, skew,        principal point and lens distortion are calibrated or determined        for each camera;    -   the tracking images from the three cameras are acquired        simultaneously such that at each time t, there are three images        taken of the element;    -   the fiducial markers are recognized in each tracking image and        the position of each fiducial marker is determined directly in        the camera co-ordinate frame;    -   the position and/or orientation of the element from the three        images is determined using a cost function to minimise the        difference in the determined position of the fiducial markers in        each of the tracking images.

29. The method according to any one or more of embodiments 16-28,wherein the medical imaging device is a cone beam computed tomographydevice.

30. A medical imaging system comprising:

-   -   a radiation source emitting a radiation beam;    -   a radiation sensor for detecting incident radiation from the        radiation beam on a sensor area;    -   an element attachable to a patient, the element comprising a        number of fiducial markers in a predefined pattern, size, shape        and/or colour;    -   at least one camera configured to take tracking images of the        element during the acquisition of medical images taken using the        radiation source and radiation sensor; and    -   computer means for determining a movement based on the tracking        images, and for adjusting the medical images acquired using the        radiation sensor to compensate for the movement.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and/or additional objects, features and advantages of thepresent invention, will be further described by the followingillustrative and non-limiting detailed description of embodiments of thepresent invention, with reference to the appended drawings, wherein:

FIG. 1 shows a flow chart of a method according to an embodiment of thisinvention.

FIG. 2 shows a flow chart of a method according to another aspect ofthis invention.

FIG. 3 shows an element according to an embodiment of this invention.

FIG. 4 shows a medical imaging system according to an embodiment of thisinvention.

FIG. 5 shows a system according to an embodiment of this invention.

DETAILED DESCRIPTION

An embodiment of the method disclosed herein is shown in FIG. 1.

In step 101, a set of at least 2 medical images of the patient, areobtained. In step 102, at least 2 tracking images are obtained. Thetracking images are images of an element that was attached to thepatient during the taking of the medical images. The tracking images andthe medical images can be related, because the tracking images and themedical images are time stamped. In step 103, any movement of theelement between acquisition of the tracking images is determined usingcomputer processing means. In step 104, the determined movement of theelement is used to compensate for any movement of the patient betweenthe acquisition of the medical images, and a digital medical model isgenerated using the adjusted medical images.

FIG. 2 shows a flow chart representing an embodiment of the methoddisclosed herein. In step 201, an element, here in the form of a plate,with at least one fiducial marker is attached to the head of a patient.The fiducial markers may be any shape, for example a circle, triangle,ellipse, or any other geometrical shape. In step 202, the patient isplaced in a medical imaging device, for example a CBCT scanner. In step203 a, the medical imaging device acquires medical images of thepatient. Concurrently with step 203 a, in step 203 b, tracking images ofthe plate are taken using one or more cameras that are placed in a knownspatial relationship with the medical imaging source and sensor. Thecameras may be integrated into the medical imaging device, or they maybe a separate system. In step 204, the position, size and tilt of thefiducial markers is determined. This can for example be done by usingprincipal component analysis. If, for example the fiducial markers arein the form of circular dots, when there is an angle between a normalvector of the plate and a linear axis between the plate and the camera,the circular dots will look slightly deformed in the tracking image. Inthis case, principal component analysis can be used to determine whetherwhat is observed in the image is a dot, and where the center of the dotis located. In step 205, a mask of the known predefined pattern of thefiducial markers is loaded from a database, and compared with thedetermined pattern of fiducial markers in each tracking image. Thiscomparison can be done using any method known in the art. This allowsthe position and orientation of the plate to be determined. It may beadvantageous to determine the orientation of the midpoint of the plate,since this will allow the highest accuracy. However, the position andorientation of any point on the plate may be used, for example thecorner of the plate. If there is more than one camera, a tracking imagefrom each camera will be taken at each time t. Each of these trackingimages will then have a determined position and orientation of the plateat each time t. The position and orientation determined from eachtracking image at time t may be slightly different because of theparticular geometry of the situation, for example one camera may have amore acute angle towards the plate than another. The determined positionand orientation from each tracking image at time t may then be combinedinto a single determined position and orientation. This combination canfor example be done by performing a weighted average of the position andorientation measurement from each tracking image at time t.

The weighted average can for example be computed by starting with thefound position and orientation of the element from one image,determining the difference between this starting position and theposition and orientation of the element in each of the other two images,and iteratively adjusting the starting position and orientation of theelement to an adjusted position and orientation, until the combinederror or difference between the position and orientation of the elementin each image and the adjusted position and orientation is minimized.

Alternatively, the starting position and orientation of the elementcould be a standard default position and orientation, and the differencebetween this standard position and orientation and the position andorientation determined in each of the three images can be computed. Thenthe starting position and orientation of the element can be iterativelyadjusted until the combined error or difference between the position andorientation of the element in each image and the adjusted startingposition is minimized. Therefore the accuracy of the determined positionand orientation of the plate will be better when more than one camera isused.

An alternative approach to the comparison step 205 may be accomplishedas follows. Instead of having a database containing a mask of the knownpredefined pattern of the fiducial pattern or markers, there may insteadbe a classification of the indices of each of the fiducial markers, asexplained in relation to FIG. 3. In this way, the 3D position andorientation of the element is then found such that the classificationindices of the known pattern is matched with the determined indices ofthe fiducial markers on the image sensor after projecting. Here it isimportant to note that the field of view of each camera, should be largeenough to unambiguously determine which part of the element is in theimage. In the case of more than one camera, there may be ambiguities asto the exact position and orientation of the element as determined fromthe tracking images taken with different cameras. In this case, a costfunction may be used, so that the position and orientation determinationis optimized using information from all cameras. In step 206, themovement of the plate between different times t is determined, and thedetermined movement of the plate is used to adjust the position and/ororientation of each pixel or voxel in a digital medical model. Since thepositional relationship between the cameras and the medical imagingsource and sensor is known, any movement of the plate can be directlytranslated into a corresponding movement of the patient, and thismovement of the patient can be used to adjust the position and/ororientation of each pixel in the digital medical model of a part of thepatient.

In FIG. 3, an element 1 according to embodiments of this disclosure isshown. The element has the form of a rectangular plate, made of a rigidmaterial. The plate has a plurality of fiducial markers 2, in apredetermined pattern, layout or configuration. The pattern should beknown to a very high degree of accuracy, so that matching subsequenttracking images taken of the plate, can be matched with a mask of thesame pattern saved in a database. In CBCT systems today, typicalaccuracy is in the range 75-350 microns at the moment. Therefore, theaccuracy of the known placement of each fiducial marker should at leastbe within this range in order to achieve a higher accuracy in thedigital medical model. Of course, the higher the accuracy of theplacement of the fiducial markers, the more the accuracy of the digitalmedical model will be improved.

Each fiducial marker may be classified using a classification index. Forexample, the fiducial marker closes to one corner could be defined ashaving the index (0,0), the next one in the same row could have theindex (0,1) and in general the fiducial markers could have an indexdefined as (i,j), with I going from 0 to n, and j going from 0 to m. Inthis way, the fiducial markers will have a known classification index,which can then be compared to tracking images to match the actualpattern of the fiducial markers on the element to the fiducial markersin the tracking images.

The plate may also comprise an asymmetrical feature 3. This will make iteasier for computer algorithms to unambiguously match the pattern fromthe database to the tracking images, and therefrom derive the actualposition and orientation of the element in each tracking image. In thecase where the fiducial markers are classified using a classificationindex, the asymmetrical feature will mean that it will be easier to makesure that each tracking camera has a view of the element wherein theposition and orientation of the element in the field of view of thecamera can more easily be unambiguously derived. That is, once thefiducial markers have been segmented in the tracking images, for exampleusing PCA, they can be classified according to the classification index.If, on the other hand, the field of view of the tracking camera onlycovered an ambiguous subset of the fiducial markers, it would beimpossible to unambiguously derive the position and orientation of theelement in the tracking image.

The element may be made of any rigid material such as plastic, metal orglass. When using coated glass for the element, it is easy to print oretch the fiducial markers onto or into the surface of the element.

Although illustrated here as a rigid plate on which the fiducial markersare printed or etched, the element may also for example be a plate withholes, with lights placed underneath the holes, so that the position ofthe lights can be picked up by a sensor. The lights could for instanceuse infrared wavelengths, and the sensor could be an infrared sensor.Another option could be to have an active plate where lights are placedon the surface of the plate, and the position of these lights could bepicked up by a sensor. For example, the light could be LED lights.

Turning now to FIG. 4, a system according to an aspect of this inventionis shown. The system comprises a medical imaging device in the form of aCBCT scanner 10, where the CBCT scanner comprises a sensor 11, and aradiation source 12. The sensor and/or the radiation source are able toturn substantially around a full circle around the patient's head. Thesystem may also comprise a chin rest 13 for the patient to rest his/herchin. The system may also include a face scanner (not shown), the facescanner configured to record a 3D model of the patient's face. Thesystem further comprises an element 1, here shown as a plate attachableto the patient's head. Also comprised in the system is one or morecameras, for example located inside the ring 15. The cameras should bemounted with a known geometrical relationship to the sensor 11 andradiation source 12. Often, this will be near or in the center of thering 15, since the patient will usually be positioned underneath thecenter of the ring 15. The cameras are configured to be used to taketracking images of the element 1 simultaneously with the medical imagingdevice taking medical images.

FIG. 5 shows a schematic of a system according to an embodiment of theinvention. The system 500 comprises a computer device 502 comprising acomputer readable medium 504 and a microprocessor 503. The systemfurther comprises a visual display unit 505, a computer keyboard 506 anda computer mouse 507 for entering data and activating digital buttonsvisualized on the visual display unit 505. The visual display unit 505can be a computer screen, or a tablet computer, or any other digitaldisplay unit.

The computer device 502 is capable of obtaining medical images recordedwith one or more medical imaging devices 501 a and tracking imagesrecorded by one or more cameras 501 b. The obtained medical images andtracking images can be stored in the computer readable medium 504 andprovided to the processor 503. In some embodiments system 500 may beconfigured for allowing an operator to control the medical imagingdevice using the computer device 502. The controls may displayeddigitally on the visual display unit 505, and the user may control themedical imaging device, as well as the tracking cameras using thecomputer keyboard 506 and computer mouse 507.

The system may comprise a unit 508 for transmitting the medical images,the tracking images and/or the digital medical model via the internet,for example to a cloud storage.

The medical imaging device 501 a may be for example a CBCT unit locatedfor example at a dentist.

Although some embodiments have been described and shown in detail, theinvention is not restricted to them, but may also be embodied in otherways within the scope of the subject matter defined in the followingclaims. In particular, it is to be understood that other embodiments maybe utilized and structural and functional modifications may be madewithout departing from the scope of the present invention.

In device claims enumerating several means, several of these means canbe embodied by one and the same item of hardware. The mere fact thatcertain measures are recited in mutually different dependent claims ordescribed in different embodiments does not indicate that a combinationof these measures cannot be used to advantage.

A claim may refer to any of the preceding claims, and “any” isunderstood to mean “any one or more” of the preceding claims.

The term “obtaining” as used in this specification may refer tophysically acquiring for example medical images using a medical imagingdevice, but it may also refer for example to loading into a computer animage or a digital representation previously acquired.

It should be emphasized that the term “comprises/comprising” when usedin this specification is taken to specify the presence of statedfeatures, integers, steps or components but does not preclude thepresence or addition of one or more other features, integers, steps,components or groups thereof.

The features of the method described above and in the following may beimplemented in software and carried out on a data processing system orother processing means caused by the execution of computer-executableinstructions. The instructions may be program code means loaded in amemory, such as a RAM, from a storage medium or from another computervia a computer network. Alternatively, the described features may beimplemented by hardwired circuitry instead of software or in combinationwith software.

1. A method for improving the accuracy of a digital medical model of apart of a patient, the medical model comprising at least two medicalimages recorded with a medical imaging device, the method comprising:obtaining a set of at least two medical images of the patient, where anelement comprising a predefined geometry or predefined information wasin a stationary relationship with the patient during the recording ofthe medical images; obtaining at least two tracking images taken with atleast one camera having a known positional relationship relative to themedical imaging device, said tracking images depicting at least part ofthe element; determining any movement of the element between acquisitionof the at least two tracking images; and generating the digital medicalmodel from the acquired medical images, wherein the determined movementof the element is used to compensate for any movement of the patientbetween the acquisition of the medical images; wherein compensating forany movement of the patient between the acquisition of the medicalimages comprises: associating a time stamp with each of the medicalimages, and each of the tracking images; determining the position andorientation of the element at each time stamp and determining therefromthe movement of the element during medical image acquisition; generatingthe digital medical model from the acquired medical images, wherein thegenerating of the digital medical model comprises iteratively adjustingthe digital medical model to account for the movement of the elementduring medical image acquisition. 2-30. (canceled)
 31. The methodaccording to claim 1, wherein the predefined information of the elementcomprises at least one fiducial marker, such as a plurality of fiducialmarkers in a predefined pattern, size, shape or colour.
 32. The methodaccording to claim 1, wherein the predefined information of the elementcomprises the geometric shape of the element.
 33. The method accordingto claim 1, wherein compensating for any movement of the patient betweenthe acquisition of the medical images comprises: associating a timestamp with each of the medical images, and each of the tracking images;determining the position and orientation of the element at each timestamp and determining therefrom the movement of the element duringmedical image acquisition; adjusting the position of each pixel or voxelof the acquired medical image with an amount corresponding to themovement of the element.
 34. The method according to claim 1, whereinthe coordinate system used in determining the position and orientationof the element can be fixed arbitrarily.
 35. The method according toclaim 1, wherein the tracking images taken and the medical images aretime stamped using the same clock.
 36. A method for improving theaccuracy of a digital medical model of a part of a patient, the medicalmodel comprising at least two medical images recorded with a medicalimaging device, the method comprising: obtaining a set of at least twomedical images of the patient, where an element comprising a predefinedgeometry or predefined information was in a stationary relationship withthe patient during the recording of the medical images; obtaining atleast two tracking images taken with at least one camera having a knownpositional relationship relative to the medical imaging device, saidtracking images depicting at least part of the element; determining anymovement of the element between acquisition of the at least two trackingimages; and generating the digital medical model from the acquiredmedical images, wherein the determined movement of the element is usedto compensate for any movement of the patient between the acquisition ofthe medical images; wherein determining the position and orientation ofthe element at each time stamp comprises: recognizing a plurality of theindividual fiducial markers in each tracking image; obtaining a digitalrepresentation in a database of the known predefined pattern or shape ofthe fiducial markers; recognizing the pattern of the fiducial markers ineach image to achieve a best fit to the known predefined pattern of thefiducial markers on the element from each tracking image.
 37. A methodfor improving the accuracy of a digital medical model of a part of apatient, the medical model comprising at least two medical imagesrecorded with a medical imaging device, the method comprising: obtaininga set of at least two medical images of the patient, where an elementcomprising a predefined geometry or predefined information was in astationary relationship with the patient during the recording of themedical images; obtaining at least two tracking images taken with atleast one camera having a known positional relationship relative to themedical imaging device, said tracking images depicting at least part ofthe element; determining any movement of the element between acquisitionof the at least two tracking images; and generating the digital medicalmodel from the acquired medical images, wherein the determined movementof the element is used to compensate for any movement of the patientbetween the acquisition of the medical images; wherein determining theposition and orientation of the element at each time stamp comprises:recognizing a plurality of the individual fiducial markers in eachtracking image; using a classification of the indices of the fiducialmarkers; and matching the known pattern of the fiducial markers on theelement to the pattern of the fiducial markers on the tracking imageusing the classification of the indices of the fiducial markers.
 38. Themethod according to claim 37, wherein recognizing the fiducial markerscomprises performing principal component analysis on the tracking imagesin order to segment the fiducial markers.
 39. The method according toclaim 37, wherein there are three cameras for recording the movement ofthe element, such as three cameras placed at the points of anequilateral triangle.
 40. The method according to claim 39, wherein: thetracking images from the three cameras are acquired simultaneously suchthat at each time t, there are three images taken of the element; thefiducial markers are recognized in each image; the pattern of thefiducial markers is recognized in each tracking image to achieve a bestfit to the known predefined pattern of the fiducial markers on theelement from each tracking image; the position and orientation of theelement in each of the three tracking images each time t is determined;and a weighted average of the position and orientation of the elementfrom the three images is used to determine the best fit for the positionand orientation of the element at each time t.
 41. The method accordingto claim 39, wherein: the camera position and rotation of each camera iscalibrated or determined; the intrinsic parameters such as the focallength, skew, principal point and lens distortion are calibrated ordetermined for each camera; the tracking images from the three camerasare acquired simultaneously such that at each time t, there are threeimages taken of the element; the fiducial markers are recognized in eachtracking image and the position of each fiducial marker is determineddirectly in the camera co-ordinate frame; the position or orientation ofthe element from the three images is determined using a cost function tominimise the difference in the determined position of the fiducialmarkers in each of the tracking images.
 42. The method according toclaim 37, wherein the digital medical model is generated in real time.43. The method according to claim 37, wherein the medical imaging deviceis a cone beam computed tomography device.
 44. A method for improvingthe accuracy of a digital medical model of a patient, the digitalmedical model being computed based on images recorded with a medicalimaging device, the method comprising: obtaining a first tracking imagetaken with at least one camera having a known positional relationshiprelative to the medical imaging device, said tracking image depicting atleast part of an element comprising a predefined geometry or predefinedinformation; obtaining a first medical image of the patient, where anelement comprising a predefined geometry and/or predefined informationwas in a stationary relationship with the patient during the recordingof the medical image; obtaining at second tracking image taken with theat least one camera having a, said tracking image depicting at leastpart of the element; determining any movement of the element betweenacquisition of the two tracking images; and generating the digitalmedical model wherein the determined movement of the element is used tocompensate for any movement of the patient relative to the medicalimaging device during the acquisition of the medical image; whereincompensating for any movement of the patient between the acquisition ofthe medical images comprises: associating a time stamp with each of themedical images, and each of the tracking images; determining theposition and orientation of the element at each time stamp anddetermining therefrom the movement of the element during medical imageacquisition; generating the digital medical model from the acquiredmedical images, wherein the generating of the digital medical modelcomprises iteratively adjusting the digital medical model to account forthe movement of the element during medical image acquisition.
 45. Themethod according to claim 44, wherein the predefined information of theelement comprises at least one fiducial marker, such as a plurality offiducial markers in a predefined pattern, size, shape or colour.
 46. Themethod according to claim 45, wherein the predefined information of theelement comprises the geometric shape of the element.
 47. The methodaccording to claim 44, wherein compensating for any movement of thepatient between the acquisition of the medical images comprises:associating a time stamp with each of the medical images, and each ofthe tracking images; determining the position and orientation of theelement at each time stamp and determining therefrom the movement of theelement during medical image acquisition; adjusting the position of eachpixel or voxel of the acquired medical image with an amountcorresponding to the movement of the element.
 48. The method accordingto claim 44, wherein compensating for any movement of the patientbetween the acquisition of the medical images comprises: associating atime stamp with each of the medical images, and each of the trackingimages; determining the position and orientation of the element at eachtime stamp and determining therefrom the movement of the element duringmedical image acquisition; generating the digital medical model from theacquired medical images, wherein the generating of the digital medicalmodel comprises iteratively adjusting the digital medical model toaccount for the movement of the element during medical imageacquisition.
 49. The method according to claim 44, wherein thecoordinate system used in determining the position and orientation ofthe element can be fixed arbitrarily.
 50. The method according to claim44, wherein the tracking images taken and the medical images are timestamped using the same clock.
 51. The method according to claim 47,wherein determining the position and orientation of the element at eachtime stamp comprises: recognizing a plurality of the individual fiducialmarkers in each tracking image; obtaining a digital representation in adatabase of the known predefined pattern and/or shape of the fiducialmarkers; recognizing the pattern of the fiducial markers in each imageto achieve a best fit to the known predefined pattern of the fiducialmarkers on the element from each tracking image.
 52. The methodaccording to claim 47, wherein determining the position and orientationof the element at each time stamp comprises: recognizing a plurality ofthe individual fiducial markers in each tracking image; using aclassification of the indices of the fiducial markers; and matching theknown pattern of the fiducial markers on the element to the pattern ofthe fiducial markers on the tracking image using the classification ofthe indices of the fiducial markers.
 53. The method according to claim47, wherein recognizing the fiducial markers comprises performingprincipal component analysis on the tracking images in order to segmentthe fiducial markers.
 54. The method according to claim 47, whereinthere are three cameras for recording the movement of the element, suchas three cameras placed at the points of an equilateral triangle. 55.The method according to claim 54, wherein: the tracking images from thethree cameras are acquired simultaneously such that at each time t,there are three images taken of the element; the fiducial markers arerecognized in each image; the pattern of the fiducial markers isrecognized in each tracking image to achieve a best fit to the knownpredefined pattern of the fiducial markers on the element from eachtracking image; the position and orientation of the element in each ofthe three tracking images each time t is determined; and a weightedaverage of the position and orientation of the element from the threeimages is used to determine the best fit for the position andorientation of the element at each time t.
 56. The method according toclaim 54, wherein: the camera position and rotation of each camera iscalibrated or determined; the intrinsic parameters such as the focallength, skew, principal point and lens distortion are calibrated ordetermined for each camera; the tracking images from the three camerasare acquired simultaneously such that at each time t, there are threeimages taken of the element; the fiducial markers are recognized in eachtracking image and the position of each fiducial marker is determineddirectly in the camera co-ordinate frame; the position and/ororientation of the element from the three images is determined using acost function to minimise the difference in the determined position ofthe fiducial markers in each of the tracking images.
 57. The methodaccording to claim 44, wherein the medical imaging device is a cone beamcomputed tomography device.
 58. A medical imaging system comprising: aradiation source emitting a radiation beam; a radiation sensor fordetecting incident radiation from the radiation beam on a sensor area;an element attachable to a patient, the element comprising a number offiducial markers in a predefined pattern, size, shape or colour; atleast one camera configured to take tracking images of the elementduring the acquisition of medical images taken using the radiationsource and radiation sensor; and computer means for determining amovement based on the tracking images, and for adjusting the medicalimages acquired using the radiation sensor to compensate for themovement; wherein compensating for any movement of the patient betweenthe acquisition of the medical images comprises: associating a timestamp with each of the medical images, and each of the tracking images;determining the position and orientation of the element at each timestamp and determining therefrom the movement of the element duringmedical image acquisition; generating the digital medical model from theacquired medical images, wherein the generating of the digital medicalmodel comprises iteratively adjusting the digital medical model toaccount for the movement of the element during medical imageacquisition.